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Abstract 

T raditional methods of paying and evaluating teachers in the United States are 
longstanding, but discussions about changing these systems to support teacher 
guality and student achievement goals are becoming more common. Efforts to 
make significant changes to these programs can be difficult and take many 
years to design, gain approval, and implement; thus, few examples of alternative 
teacher compensation and evaluation systems exist. Relieved from many of the 
restrictions and reguirements associated with most traditional public school 
systems, charter schools often are better positioned to implement changes 
more guickly than a traditional school, yet their experiences can provide useful 
information for others who are engaging in similar activities. Thus, the 
experience of a large urban charter school that designed and implemented an 
innovative teacher compensation system and standards-based teacher 
evaluation system that has been in place for six years offers important lessons 
in designing, implementing, evaluating and refining these systems. 


The Vaughn Next Century Learning Center has been one of the CPRE-UW 
Teacher Compensation Project’s research sites since 1998, when the school first 
implemented knowledge and skill-based pilot program and a school-based performance 
award program. The purpose of this paper is to provide a brief overview of the school’s pay 
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and evaluation programs, their evolution and current status, and to highlight findings from 
the employee attitude surveys that CPRE-UW researchers administered to Vaughn teachers 
the past five years. Finally, the paper will note key themes that may be of use to others who 
are considering compensation or evaluation reforms. Although other areas of research and 
inquiry are logical extensions to the work reported here, this paper primarily provides a 
longitudinal review of one school’s teacher compensation and evaluation innovation, 
including design, implementation, and evaluation, drawing from multiple information 
sources at the school and individual level. 

Background 

The Vaughn Next Century Learning Center is a public conversion charter school in 
the Los Angeles Unified School D istrict. The school became a charter school in 1993 after a 
majority of the school's teachers petitioned for that status. Each five-year charter includes 
specific outcomes that will be achieved during the charter period and those outcomes inform 
virtually every decision made at Vaughn. The current principal, Yvonne Chan, has been with the 
school since 1990 and was transferred there prior to the school's conversion. Chan is widely 
credited with a dynamic leadership and inclusive style that has contributed to improved student 
achievement. The school's immediate neighborhood has been somewhat revitalized by the 
school's creation of a "compound" that includes a Family Center, a "one-stop shop" that 
provides families with basic needs, such as prenatal care, family literacy, adult education classes, 
job referrals, and services provided in collaboration with various agencies. The school's current 
charter includes plans to add one additional grade level a year so that the school eventually will 
serve students pre-K through high school; grade seven was added for the 2003-04 school year. 

Vaughn employs about 75 classroom teachers and 50 paraprofessional and 
certificated support staff. Maximum class size is kept at 20 students (PreK-5) and 25 students in 
middle school. Every family is expected to be involved in the school community in some way, 
including signing an annual home- school compact that the parents will spend at least 30 hours a 
school year in activities that support their children's education. The school's governance 
structure includes roles for teachers, administrators, and parents, reflecting that responsibility 
and accountability for the education of Vaughn's students is shared and that everyone in the 
school community has a stake. It was from within the context of accountability and desired 
outcomes that the performance pay and teacher evaluation innovations arose. 

The development of the knowledge- and skill-based pay program began during the 
1997-98 school year. Six main issues and concerns were identified that contributed to the 
decision to develop a new pay system. They were: 1) pay inequity among teachers, 2) a concern 
that the single salary schedule promoted mediocrity, 3) a perceived need to develop a more 
explicit internal accountability focus to support the external accountability required by the 
school's charter, 4) a need to retain and incent new teachers, 5) a concern that many classroom 
teachers perceived movement into administrative positions as the only way to make more 
money, and 6) an interest in making improvements to teacher's salaries directly related to 
increased student achievement (Kellor, Milanowski, O dden, & G allagher, 2001). 

Implementation of the pilot knowledge and skill-based pay system began in 1998-99 for newly 
hired teachers and for current V aughn teachers within their first five years of employment at 
Vaughn and volunteered to participate. A school-based performance award program covering 
all teaching staff also was implemented that year. 
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An essential piece of a knowledge and skill-based pay system is a way to measure the 
knowledge and skills to be compensated, thus, a new evaluation system was implemented 
concurrent with the first year of the new pay system. The first year it was used only for those 
teachers who were covered by the new pay system, and was expanded in subseguent years to 
include all teachers so that all teachers would be held to the same teaching expectations. The 
new system was loosely based on Charlotte D anielson's F ramework for T eadiingi(D anielson, 1996), 
which defines four broad "domains" of teaching, with specific attributes defined under each of 
the domains. D anielson's framework includes four performance levels, Unsatisfactory (1), Basic 
(2), Proficient (3), and D istinguished (4), with a description provided at each level for every 
performance component. 

A unigue aspect of how Vaughn developed and implemented its new pay program 
and evaluation system is that the programs intentionally were implemented before all the details 
were resolved. In the view of the principal, they were "designing the plane as they were heading 
down the runway." Both the staff who were designing the programs and the governance 
committees that were responsible for approving the programs believed that a guick 
implementation was necessary to show commitment to the new programs and to get a start on 
achieving the positive outcomes, including improved student achievement that they expected 
would flow from the new pay and evaluation programs. Thus, important policy and conceptual 
decisions were made and the programs were implemented "as is," with a public commitment to 
revise and refine as experience was gained. 

Vaughn has continued to "tweak" the programs annually and sometimes during the 
school year, based on internally- identified needs and issues and in response to information 
gleaned from annual employee attitude surveys conducted by T eacher Compensation 
researchers at the University of Wisconsin-Madison branch of the Consortium for Policy 
Research in Education (CPRE-UW). It included guestions relating to both the pay program and 
the evaluation system. Although direct comparisons over time can't be made because the 
guestions are not identical across all years, some general observations and themes based on the 
responses will be noted later in this paper. (A copy of the most recent version of the survey is 
available upon reguest from the author.) 

Overview of the Pay and Evaluation Programs 

This section includes a summary of these programs for each of the past six years. 
Because the school-based performance award program underwent few changes after the second 
year, that program is summarized first in a separate section, followed by an overview of the 
evolution of the knowledge- and skill-based pay and evaluation programs. 

School-Based Performance Award: A Bridge Between Old and N ew 

Although the school-based performance award (SBPA) program was created as an 
adjunct to the school's knowledge- and skill-based pay program, the initial compensation design 
did not include a SBPA program. Early in the design process, senior teachers appeared to view 
the new pay structure development as a personal threat o r as implying that V aughn intended to 
pay its teachers less. In January 1998 a draft plan was presented to V aughn's Business 
Committee, which is responsible for teacher evaluation and salary determination. The 
submission of the draft plan brought senior teachers more directly into the discussion and 
review of options. In May a task force of newer and veteran teachers met to work out more of 
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the details of the new pay program, and that group raised the guestion of why all teachers 
would not be compensated for those items included in the draft pay system that were based on 
specific outcomes at either the school-wide, grade-level, or classroom level. 

The program designers then decided to include a SBPA element as part of its new 
compensation program to support the school- wide goal of improved student achievement and 
to function as a bridge between those teachers on the traditional salary structure and those in 
the new pay system. The SBPA also was intended to strengthen the concept of internal 
accountability by linking some form of compensation directly to student achievement for all 
teachers. 

The initial SBPA program included two elements of performance: school-wide 
student achievement and school- wide redesignation of limited English proficiency students to 
English language learners (ELL). These two performance measures were key parts of the 
school's charter plan for 1998-2003 and thus were chosen as the SBPA's performance measures. 
D ue to difficulties in establishing a measurement method and goal and the introduction of a 
new statewide accountability program beginning the 1999-00 school year, the redesignation 
portion was never implemented. 

For 1998-99, the school used three measures of student achievement: 1) school-wide 
improvement on the state- mandated Stanford 9 (SAT9) reading, math and language tests for 
grades 1-5; 2) school- wide performance on the Terra Nova (English and Spanish versions) 
reading and math tests for grades K -5; and 3) school- wide report card grades. T o receive the 
school- wide performance award the school had to post an increase of at least three percentile 
points over the previous year's SAT9 school-wide average score and meet at least one of the 
other two achievement goals: an average school- wide score of 37 th percentile or higher on the 
T erra N ova post-test or a school- wide average report card grade of "C" or better. D ue to 
concerns about public pressure and credibility and potential problems in relying on a single 
measure, the school chose a combination of standardized test scores and grade level 
performance, as measured by report card grade. Compared to some SBPA programs that use a 
combination of academic and non- academic measures such as attendance, Vaughn's plan was 
relatively straightforward. The school-wide student achievement goals were met for 1998-99 and 
all certificated staff, including administrators, received $1,500. 

After the first year, Vaughn continued its SBPA program but modified it in response 
to a bonus award component of the State's new accountability program. A key piece of the 
Public Schools Accountability Act of 1999 reguired that the California D epartment of 
Education annually calculate an Academic Performance Index (API) for every public school in 
California, including charter schools, and to publish school rankings based on the APIs. The 
Act also established a minimum five-percent annual growth target for each school. Schools that 
met API growth targets could be eligible for awards under several state programs, with potential 
individual awards of up to $25,000. Rather than operate a separate program, V aughn decided to 
tie its SBPA program directly to the API program, with eligibility for a SBPA dependent on the 
school meeting its API growth target in a particular year. D ue to changes in the assessment tools 
and the methodology to integrate results from multiple assessment measures, the actual 
calculation of the API varied somewhat from year to year and thus is not included in this paper. 
(California D epartment of Education, 2003) 

Vaughn's growth exceeded the target goal in each of the years for which API data 
are available. Table 1 shows the API base, growth target, and actual growth from 1999-00 
through 2002-03: 
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Table 1 

API base, growth target, and actual growth from 1999-00 through 2002-03 


School year 

API base 

G rowth target 

Actual growth 

1999-00 

443 

18 

51 

2000-01 

494 

15 

85 

2001-02 

591 

10 

18 

2002-03 

619 

9 

41 


Source: CDE, Policy and Evaluation Division, Academic Performance Index Growth Reports 

For most California schools, the actual payment of the state- funded bonuses was 
delayed most years by funding complications and difficulties in verifying final student 
achievement data schools. However, Vaughn's teachers received their bonus payments under 
the Vaughn SB PA program almost as soon as the test data were available, because Vaughn's 
administrators had pledged to pay the bonuses from funds already under the school's control 
rather than to wait for the state to release the API award funds. Strategically, this allowed 
Vaughn to maintain teachers' trust in the SB PA program and the school's commitment to 
deliver what it had promised. After three years at $1,500, the bonus award was increased to 
$2,000 for the 2001-02 school year and has been maintained at that level. 

Several other pay innovations are included in Vaughn's pay system. A contingency- 
pay program gives participating teachers a $250 bonus at the end of the school year for each of 
the identified areas for which the stated goal was achieved; the contingency plan was revised in 
2003-04. (Refer to Appendix A and B for details.) A "gainsharing" program was developed, in 
which participating teachers receive the savings from decreasing the use of substitutes for the 
respective teacher. The pay program also provides "expertise pay" for specific roles or 
responsibilities. In the second year, a new "stock options" program was created by investing a 
lump sum received from a new grant. A total of $290,000 was invested on behalf of all 
certificated staff participating in the performance pay program, with distribution policies still 
under development, but with the understanding that the "Employee Incentive Fund" is 
intended to be a long-term investment with no immediate payouts. 

The next section summarizes the performance pay (knowledge and skill-based pay) 
plan from its development through the sixth year of program operation. 

Evolution of the Performance Pay Program 

The first year of the performance pay program was intended to introduce the new pay 
system to new and early career teachers, who were perceived to have lesser commitment to or 
investment in the traditional salary structure than more veteran teachers. By phasing- in the new 
pay system, the school also hoped that veteran teachers would gain trust in the new system as 
well as confidence that the new system could be a positive change force. T eachers would be 
evaluated on the competencies each semester and eligible to receive half of the annual amount 
for each competency each semester. The competencies included in the performance pay 
program were chosen because in addition to directly relating to instructional principles in the 
school's charter they were viewed as non- controversial. The designers also believed that they 
could be measured by existing assessments or that new assessment tools could easily be 
developed to measure them. 
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The base pay rate for a first-year teacher was set at $30,000 for an emergency 
credentialed teacher and $31,000 for a teacher with a California elementary teaching credential. 

E ach year for the first five years a teacher would receive a base increase of $1,000, with an 
additional $1,000 annually if the teacher had a master's degree. Although the $36,000 after five 
years for a credentialed teacher was less than the rate for a five-year teacher under V aughn's 
traditional schedule, $37,867, additional pay increments available under the pilot performance 
pay system offered the potential of $3,300 more for four competencies related to instruction, 
and up to $800 more for the four contingency-based items. Potentially, a first year teacher 
would receive a base rate of $31,000 and up to $4,100 in additional pay under the performance 
pay program. (See Appendix A for a summary of V aughn's performance pay provisions for the 
first five years.) 

The plan also offered the prospect of higher total compensation for existing teachers 
who were early in their careers at V aughn. A five-year teacher would receive $36,000 in base 
pay, and up to $4,100 in performance pay, several thousand dollars more than under the 
traditional salary schedule. Five early career teachers opted into the program the first year. They 
cited a variety of reasons, including the potential for more pay, the eguity of a pay system that 
did not restrict pay opportunities for those who did not have the time and / or money to take 
advanced coursework needed for salary advancement under the traditional system, and to be a 
role model by being held accountable for student achievement. 

Although the original plan was to gradually expand the pilot pay program over 
several years to include more senior teachers, changes were made effective the second year that 
would allow more senior teachers to opt in. This change in direction was in large part a response 
to demands by some veteran teachers that they be able to participate in the pay program after 
they saw the amount of performance pay that many of the teachers in the pilot program 
received. These teachers believed that they were already performing at the level that would be 
reguired to receive the additional pay and that they could increase their compensation without 
significant additional effort or change in their performance. The second force behind the earlier 
expansion related to changes at the state level in grant and other funding opportunities that 
Vaughn could receive for certain instructional changes, along with state- reguired changes to the 
English as a Second Language (E SL) instruction that would reguire all Vaughn staff to be more 
involved in and responsible for E SL instruction. Thus, effective with the 1999-00 school year, 
the performance pay program was offered to all Vaughn teachers regardless of their length of 
service at the school. Participation for new hires continued to be mandatory, and teachers who 
opted into the new program could not switch back to the traditional salary schedule. 

The design team for the expanded pay program was essentially an ad hoc group that 
included some of the same people who designed the pilot, joined by some veteran teachers who 
wanted to help make decisions about a revised program that could impact and appeal to more 
veteran teachers. The principal was involved in the design process, but tried to give the teachers 
the lead role. She stated that after the fact, this might not have been the best strategy, because it 
might have put the teachers in an unreasonably awkward position by reguiring them to deal 
directly with their peers on controversial and sensitive pay issues. The potential availability of 
state funds earmarked for specific educational reform goals also meant that the pay designers 
had to consider whether the state funds were related to the school's charter and if so, how they 
could best be used to support the new pay program. This reguired that the design team deal 
with school-wide issues of curriculum design and delivery, but did give the school access to 
some additional funds that were used to enhance the pay program. 

The design of the initial pilot program was not without challenges and new issues 
continued to emerge as the designers sought to expand the program. The primary challenge was 
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to define pay principles in a way that would not seriously disadvantage veteran teachers, 
particularly those near the top of the traditional salary structure. Pay plan designers knew that if 
movement to the new pay system reguired too large a financial penalty, teachers would be 
unlikely to take the risk. A second challenge arose as the potential value of the performance pay 
was considered. For teachers in the pilot program, the potential performance pay available was 
almost 25 percent of their base pay. That same dollar amount, however, was only 5-7 percent 
more than the base pay of a more veteran teacher at or near the top of the salary schedule. 

Thus, a fundamental guestion arose: should the potential pay be "indexed" to the base salary 
rate, so that the same potential differential was available to all teachers, or should the 
performance pay amounts remain a flat dollar amount— providing the same amount of 
compensation for the same demonstrated performance. Estimates of veteran teachers' pay 
under the new system suggested that the proposed pay system would net veteran teachers on 
average the potential for 17 percent more pay than they were receiving under the traditional pay 
system. 

While recognizing the veterans' concerns that a new pay system should not drastically 
affect their "guaranteed" base salary, the design team believed that the new pay system must 
strike a balance between that concern and the need for a new pay system to foster and be 
supportive of individual accountability. The designers also believed that for the new pay system 
to have a motivational effect, the potential payout under the new plan, and the potential cost of 
not performing well enough to earn at least some of the performance pay, had to be considered. 
T o help address the tension between the need to decrease base pay rates under the new system 
and the desire to make the new plan attractive to teachers, the plan's designers adopted a policy 
statement that whatever base rates were decided, monthly "off-sets" would be available to help 
veteran teachers make the transition to the new and likely lower base pay rate. 

Another challenge to the expansion of the pay program was whether there should be 
additional or different knowledge and skills identified for experienced teachers. The designers 
agreed that the initial set of competencies should apply to all teachers, and that additional 
competencies should be identified that would presumably be applicable to more experienced 
teachers. The result was the creation of three levels. Level 1 was slightly reconfigured for the 
second year and deemed "depth in essentials," those knowledge and skills that all teachers at all 
levels of experience should be able to demonstrate in their teaching. Level 1 included: literacy, 
ESL / Sheltered English, technology, special education inclusion, classroom management, and 
lesson planning. Level 2 included four additional competencies (mathematics, social studies, 
science, arts) that represented the breadth of knowledge and skills reguired of all teachers who, 
as elementary school teachers, essentially function as generalists and are responsible for 
instruction in all content areas. Level 2 also included English learner's support (BCLAD ), with 
three possible strands of expertise. To be eligible for any of the Level 2 performance pay, a 
teacher must have received an average rating of 3.0 on all Level 1 competencies and have a clear 
California teaching credential. The latter reguirement also was intended to serve as an additional 
incentive for emergency credentialed teachers to obtain a clear credential. 

Level 3 did not include any additional knowledge or skill areas, but rather reguired an 
overall level of teaching performance markedly higher than that reguired for Level 2 
performance pay eligibility. To be eligible for Level 3 pay, a teacher must have received an 
average rating of at least 3.5 in all skill areas (Levels 1 and 2). Thus, an experienced teacher with 
above average teaching performance could earn up to $13,100 in additional pay under the new 
pay system ($3,500 for Level 1, $5,600 for Level 2, and $4,000 for Level 3). 

The decision whether to stay with the traditional pay system or opt in to the new system 
seemed to reveal two disparate views among the veteran teachers. Some teachers saw the new 
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pay system as a way to create a more equal playing field that compensated people in a manner 
that was more directly tied to their actual contributions to Vaughn and its students, rather than 
on how well they had "played the game" of taking advanced courses and acquiring degrees. 
Other teachers who had successfully moved up the traditional salary schedule felt they would be 
penalized for having played the game by the rules in place at the start and believed they should 
in essence be held harmless in terms of base salary rate, and not denied the opportunity to 
receive performance pay "bonuses." Ultimately, the pay program included annual base pay 
increases up to the 11 th year, and then an increase at year 15. The expanded pay program also 
modified the criteria for the Master's degree add-on, making it available to people who had 
obtained 30 educational units after attaining a clear California teaching credential but not a 
Master's. This was intended to help address the concern of some veteran teachers that the 
additional coursework they had taken would be completely disregarded under a new pay system. 

The beginning of Year 3 (2000-2001) marked completion of a year of experience with 
the expanded pay program and two years with the program's first cohort of newer teachers. 
Some relatively minor changes were made to the amount of performance pay for some 
competencies to more accurately reflect the relative value of the respective knowledge and skills. 
In addition, Mathematics was moved from Level 2 to Level 1 and T echnology was moved from 
Level 1 to Level 2. These switches were made because after additional consideration, 
mathematics was viewed as a critical area that should be included at Level 1 to stress its school- 
wide importance and fundamental place in the school's curriculum. T echnology's movement to 
Level 2 was due to ongoing problems with making technology tools regularly available to all 
teachers to use in their day-to-day instruction, as well as a reassessment of the relative 
importance of Technology to the school's overall goals. 

The fourth year (2001-2002) the pay program saw few changes, and it was 
unchanged for the 2002-03 school year. Physical Education was added to Level 2, the amount 
paid for attaining Level 3 was increased, and a fourth level, D istinguished T eacher, was added, 
which would pay $2,000 to someone after five consecutive years of attaining Level 3 
performance. For the fourth year, all but five veteran teachers had opted into the performance 
pay program. By the end of that year, three of the veteran teachers who had remained on the 
traditional salary system planned to return to Vaughn the following year; one other planned to 
retire and one to return to a LAUSD position Ultimately, of the three, one stayed and converted 
to performance pay, one switched to performance pay mid-year, and one moved out of town. 
Thus, by the fifth year, the traditional teacher salary schedule became part of Vaughn's history 
and all teachers, current and future, were included in the performance pay system. 

The sixth year (2003-04) saw more significant changes to the performance pay 
structure. The addition of middle school staff who are subject specific teachers rather than 
generalists, and the school's goal to have one pay system applicable to all teachers, had made it 
difficult to use the subject- specific competencies of the former system. The school also felt that 
because most staff were earning the performance pay bonuses in every Level 1 competency 
area, the pay associated with those competencies should be reallocated to base pay, and the 
performance pay program should focus on more general instructional areas. Thus, although the 
pay program retained three levels, they became tied to performance on four domains— very 
similar to the original domains in Charlotte D anielson's F ramework for T eadring— rather than to 
content- specific competencies. The total amount of performance pay potentially available to a 
teacher is $13,050; the revised pay schedule is shown in Appendix B. 
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Evolution of the T eacher Evaluation System 

Teacher evaluation at Vaughn prior to implementing the performance pay system was 
taken seriously, although it was not tied directly to pay. T eacher evaluation responsibilities were 
a significant part of two administrators' jobs and V aughn teachers were accustomed to being 
observed and evaluated. The development of a new teacher evaluation system at the same time 
the new pay plan was being implemented meant that there were only two months to 
communicate and educate teachers about the new evaluation system, conduct observations and 
make decisions regarding whether a competency had been earned. 

The determination as to whether an individual had demonstrated a competency was 
based on the average of the teacher's self-evaluation, a peer evaluation, and an administrator's 
evaluation. The four- level rating scale (1, 2, 3, 4) from D anielson's F ramework for T eaching was 
used, but was open to individual interpretation; there were no rubrics or guidelines to help 
evaluators determine the rating level. Instances where there were obvious discrepancies between 
the three ratings were handled on a case- by-case basis, e.g., if a teacher gave a self-assessment 
that was much higher than the peer and administrator's evaluations. The discrepancy was 
brought to the teacher's attention and an opportunity given to revise the self-assessment, 
although it was not reguired that the teacher do so. To receive pay for a competency, the 
average rating had to be 2.5 or higher. 

Concerns emerged about the evaluation system during the pilot year. The generic four- 
level rating scale was not adeguate to ensure either clear understanding of the expectations for 
each competency or for consistent rating by all evaluators on performance levels within each 
competency. The school committed to develop rubrics for each competency area for each level 
for the second year of the performance pay pilot. The main goal of more detailed rubrics was to 
provide clearer guidance to both teachers and evaluators at all points of the evaluation process. 

Beginning with the 1999-2000 school year, all teachers were evaluated under the new 
evaluation system, not just those teachers in the performance pay program. This was also 
intended to encourage more teachers to opt into the performance pay plan, because if they were 
going to be evaluated under the same system as those on performance pay, they would probably 
want to be eligible for the same pay enhancements. By the end of the first semester of the 
second year, over $107,000 was paid to teachers in the knowledge- and skill-based pay program, 
including payments of over $5,000 to several veteran teachers who attained the Level 3 rating 
and thus received all of Level 1 and Level 2 pay, as well as the Level 3 pay. 

Some operational and administrative changes were made to the evaluation system in 
the third year. Probably the most visible one was the creation of a specific committee, the 
Performance Assessment Review (PAR) Committee. Rather than relying on an ad hoc group of 
people with no clearly defined set of responsibilities or relationship to the school's governance 
structure, this committee was formally charged with ongoing responsibility to work on the 
evaluation system, including broad policy issues, as well as the administrative procedures and 
process. 

As a result of its work that year, numerous changes were made to the evaluation 
system, including creating a pre- and post-conference system, an appeals process for 
evaluations, methods for evaluating team teachers who do not teach all content areas, and 
addressing inter- rater reliability. CPRE-UW interviews with teachers near the end of that school 
year revealed that they clearly viewed the PAR Committee and its responsibilities as important 
to the overall successful functioning of the evaluation system. The school's guick response to 
teachers' concerns about the evaluation system showed the importance of the evaluation system 
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to the successful operation of the performance pay program and that the school was serious 
about creating a fair and trusted evaluation process. 

A lingering concern on the part of both teachers and evaluators was the time 
reguired to accurately evaluate a teacher. The plan was to have two evaluation "windows" each 
semester, for a total of four evaluations from each evaluator each year. For each of the first core 
subject areas in which a teacher was evaluated, both the peer and administrator were to conduct 
a longer evaluation of up to 30 minutes of a specific lesson in the core area, supplemented by 
several shorter walk-throughs. D uring the subject-area observation, the evaluator would also 
look for evidence of the other competencies, e.g., classroom management or lesson planning. 

T eachers being evaluated for Level 1 competencies thus could expect evaluations in three core 
areas (Literacy; ESL / Sheltered English; Mathematics) four times a year, by two evaluators. For 
Level 2 or 3, total observation time in the five core areas (Social Studies and Science, as well as 
the three previously noted) could be up to about two hours by two observers, four times a year. 
This became too burdensome for both the teachers and evaluators, and mid-way through the 
second semester, the fourth observation was made optional. Although most teachers waived it, a 
few reguested the fourth set of observations, especially if they thought an additional observation 
would improve their overall evaluation score. 

D espite the amount of time spent on classroom observations, teachers still had concerns 
about whether all aspects of their teaching could be evaluated accurately. The introduction of 
pre- and post-conferences was intended to help assuage some of these concerns by providing an 
opportunity for teachers to bring the evaluators' attention to aspects of the rubric that might not 
be readily observable from a routine classroom observation, and for evaluators to ask for 
evidence of aspects of teaching that were not present in a particular observation session. 

Feedback after the observation also was addressed the third year. Because Vaughn 
already had a separate mentor system for new teachers the evaluation system initially did not 
include feedback and was intended to be more of a summative than a formative review. Internal 
comments and responses to the employee attitude survey after the first two years indicated that 
teachers and evaluators both wanted to be able to give and receive more feedback as part of the 
evaluation process, and in particular, feedback about areas in which performance could be 
improved or evidence provided to support items in the rubric. The post- conferences introduced 
the third year offered a partial solution to this problem by providing a specific time where the 
feedback could be provided. The evaluation system also was modified to allow evaluators to 
provide feedback about how to improve one's performance. 

D uring the fourth year (2001-02), work began on developing rubrics that would be 
needed when the middle school expansion began. The PAR Committee believed that even 
though only one grade level would be added each, beginning with sixth grade, rubrics for all 
three middle school grade levels should be developed so there would be a clear set of 
expectations for all the grade levels from the outset. There also was a belief that the teacher and 
student standards should be developed concurrently to ensure consistency and alignment 
between the standards and curriculum. A set of draft standards for middle school teachers were 
developed and were intended to be used the following school year when the school added grade 
6 . 

The subseguent year, the effort to develop performance rubrics for all grade levels 
and content areas at the middle school experienced a setback when the sixth grade rubrics 
proved difficult to administer. An interim solution was devised for the remainder of the school 
year: continuing faculty were paid on the basis of their previous year's evaluation, and the one 
new staff member was evaluated based on classroom observations by another sixth grade 
teacher and an administrator. The entire set of performance rubrics, both middle school and 
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elementary, were put under review and revision by the PAR Committee, with the middle school 
rubrics being the highest priority. New rubrics applicable to all grade levels were completed for 
use for the 2003-04 school year and paralleled the changes made to the performance pay 
program described earlier. 

T eacher Evaluation Scores and Student Achievement: Preliminary Findings 

Perhaps the most-asked guestion about any education reform is whether it improves 
student achievement. Clearly, test scores at Vaughn have risen dramatically during the last 
several years, but the role or influence of the performance pay plan or evaluation system in this 
improvement can't be presumed. There are inherent difficulties in attempting to measure any 
particular teacher's effect on student achievement, but one method that is growing in popularity 
is value-added analysis, wherein pre- and post-test results are used to identify student 
achievement growth over a specified period of time. Vaughn's standards-based teacher 
evaluation system and the availability of several years of test scores for a large number of 
V aughn's students presented an opportunity for CPRE-UW to conduct a value-added analysis 
of student achievement at the classroom level and the relationship of classroom performance, if 
any, to teachers' evaluation scores. 

In brief, the study used a sophisticated statistical technigue, hierarchical linear 
modeling, to compare subject area teacher evaluation scores and student subject area 
standardized test scores for reading, math, and language arts for thirty-four 2 nd - 5 th grade teachers 
and all of their students for whom two years of student achievement were available. T eacher 
performance was measured by using teachers' average score in each evaluation domain across 
the three observation windows in the 2000-01 school year. A pre- and post-test model was used, 
measuring value-added achievement on the SAT 9 test as administered in California. Of the 
three subject areas analyzed, a statistically significant correlation was found between the 
teachers' average score on the literacy domain and average gains in classroom level student 
achievement and positive relationships in the other two subjects. The demonstrated positive 
relationship between a teacher's evaluation score in the subject area domain and student 
achievement in that subject area is an important finding. (G allagher, 2004) 

Vaughn T eachers Reactions to Pay and Evaluation Innovations 

The introduction of a new pay system or changes to an existing pay system can cause 
turmoil in an organization, and schools are no different in that regard (Odden & Kelley, 2002). 
Because not all of Vaughn's staff embraced the idea of a new pay system, the effects of the new 
pay system on the school's culture and staff's responses towards the new pay system were of 
interest to both the school and researchers, especially since there are relatively few opportunities 
to examine the effects of compensation changes in the K-12 public school context. At Vaughn, 
the teachers' responses to a new evaluation system were of special significance because of the 
possible overlap into their perceptions and acceptance of the new pay system. Changes to an 
evaluation system also can be controversial, especially so if the evaluation outcome has some 
influence on pay or pay increases (K imball, 2002; O dden & K elley, 2002). 

The context of V aughn's new pay system was even more interesting because the impetus 
for change came primarily from staff, making it more of a bottom-up change than often is the 
case when compensation or evaluation systems are changed. Similarly, because the new 
evaluation system evolved out of a need created by the new knowledge and skill pay system, it 
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could be considered more of an "organic" change than is typical when evaluation systems 
change. Vaughn's leadership and staff were willing to allow the school to become an ongoing 
research site for the CPRE-UW Teacher Compensation Project, offering a rare opportunity to 
track the evolution of these innovations from the first year of implementation to the present. 
The school's leadership also used the results of each year's survey as a form of ongoing 
evaluation, leading to changes and improvements. CPRE-UW researchers agreed to conduct an 
employee attitude survey each year, which was supplemented in most years by on-site interviews 
of representative teachers and administrators. 

The survey was designed to address themes of interest related to introducing 
performance pay systems, as well as focusing each year on a guestion or area of interest to the 
school. The original themes identified for the survey were: 1) the new process to assess teacher 
competencies and performance levels; 2) the effect of the pay system on the school's sense of 
community and overall morale; 3) the alignment of related systems, such as professional 
development, with the demands of the new pay system, and 4) the extent to which the new pay 
system was perceived as influencing teachers' decisions to work at V aughn. 

Overall, the surveys had exceptional response rates, ranging from a low of 84 percent to 
a high of 99 percent. The surveys were distributed to staff at a staff development retreat just 
before the start of the school year so even though participation in the survey was not 
mandatory, teachers typically had work time to complete it. The school's culture supported its 
role as an innovator in education reform, and it is likely that many of the school's teachers 
shared that belief and thus were more likely to participate in the survey, knowing that the results 
could help others who were making changes to teacher pay and evaluation systems. After the 
first and each subseguent survey, the researchers prepared a report for the school. (Kellor & 
Milanowski, 2004; Kellor & Milanowski, 2003; Kellor & Milanowski, 2002, September; 
Milanowski & G allagher, 2001; Milanowski & Kellor, 2000; Milanowski & Kellor, 1999). 

School-Based Performance Award Program 

The guestions in this section remained the same each year and are relatively easy to 
summarize and note general trends. Overall, the bonus program generally was accepted guite 
well from the outset. A few guestions showed dips and rises from year to year, but there were 
no guestions for which a majority did not agree every year, and never any indication that the 
program should be abandoned. From the first year on, a majority of the respondents agreed that 
it was fair to hold teachers accountable for student achievement, with 20 percent disagreeing the 
first year and only 2-7 percent disagreeing in subseguent years. Similarly, a strong majority, 80 
percent or more each year, agreed that it is fair for teachers who increase student achievement 
to receive a bonus, with the percent agreeing going as high as 98 percent one year. 

The shift to using the API as the indicator for bonus payment eligibility beginning 
the second year did not appear to have any apparent effect on how V aughn's teachers felt about 
the performance award program. Although respondents agreed with the statement that they 
"would work just as hard to achieve the school-wide outcome goals even without the possibility 
of getting a bonus," a large majority of respondents also agreed each year that as a way to 
measure the school's performance, the school-wide outcome goals are fair; they have a clear 
understanding of the school- wide outcome goals; the bonus provides a clear goal that 
encourages school staff to work together; the possibility of a bonus provides a reason to make 
the extra effort needed to meet the goals; the possibility of a bonus motivates to work toward 
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the goals; the amount of bonus money is enough to motivate; and the school-wide bonus 
should be continued. 

These responses suggest that even though teachers indicated they would work just as 
hard to achieve the goals if there was not money associated with it, they seemed to place some 
value on the prospect of receiving a bonus. 

Two guestions relating to expectancy theory, a motivation theory, were asked the 
first four years. The first asked the probability or likelihood of the school meeting its outcome 
goals if the teacher and his / her colleagues worked together as hard as they could. The mean 
rating each year ranged from 78.3 percent to 87 percent, suggesting that there was a relatively 
high degree of confidence that the goal could be achieved. A similar guestion was asked of 
teachers who participated in school-based performance award programs in Kentucky and 
Charlotte, North Carolina. Every year, Vaughn's teachers were consistently much more positive 
about this outcome than either the Kentucky respondents (52.7 percent) or Charlotte 
respondents (62 percent). (Kelley, Heneman III, & Milanowski, 2001) 

The second guestion asked the probability or likelihood that the teacher would 
receive the pay bonus if the school met the outcome goals; the mean rating ranged from 81.1 
percent to 90 percent. In an ideal world, the response would be 100 percent— every teacher 
believing that the bonus would be received if the goal were met. Nonetheless, the responses 
from Vaughn teachers suggest a relatively high degree of confidence that the money would be 
paid, perhaps reflecting the degree of trust the staff in the school's principal and the financial 
ability of the school to pay the bonus. In contrast, teachers in Charlotte and Kentucky 
responded guite differently to a similar guestion. Although almost three- guarters (72.8 percent) 
of respondents in Charlotte believed they would receive the bonus if the goals were met, just 
over half (54.8 percent) of Kentucky respondents believed they would receive the bonus if the 
goals were met. There are many possible interpretations for the strikingly different responses in 
respondents, including size (school, district, state) and previous experiences of something being 
promised and not delivered, which was perceived as a factor contributing to Kentucky teachers' 
perception that the probability of receiving the money was only slightly better than chance. 
(Kelley et al., 2001) 

Performance Pay and T eacher Evaluation Programs 

E ach year the survey included guestions about aspects of the performance pay and 
evaluation programs. The first survey did not address evaluation issues as explicitly as did 
subseguent surveys. Responses the first year and feedback from interviews and sources within 
the school made it clear that there were strong concerns especially about the evaluation process, 
so specific guestions about the evaluation process were added to the survey beginning the 
second year. 

Responses to the survey for the first year of the new pay and evaluation programs 
(1998-1999) revealed something of a fissure between the general understanding and acceptance 
of the knowledge and skill pay plan, and how teachers felt about how it was being implemented, 
especially how the knowledge and skills were evaluated. Although a majority of respondents 
believed they had a clear understanding of the knowledge and skill areas, could explain them to 
others if asked, believed the knowledge and skills were important to good teaching at Vaughn, 
and were motivated to increase their own knowledge and skills in those areas, the positive 
responses generally did not carry over to how the knowledge and skills were evaluated or what 
supports were available to teachers to improve their knowledge and skills. 
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Some striking findings were that less than 30 percent of respondents believed that 
the knowledge and skill assessment process was valid, that the assessment process was fairly run, 
and that the way the knowledge and skills were measured was fair. Curiously, although only 
about 40 percent of the respondents said that the knowledge and skill pay greatly influenced 
what professional development the person took the previous year, more than 80 percent said 
they made a lot of effort the previous year to develop the knowledge and skills that were 
rewarded by the pay system. The mismatch between the professional development and skill 
development responses suggest that people may have made individual, personal efforts to 
improve their knowledge and skills, rather than integrating the effort into formalized 
professional development opportunities. However, almost three- guarters (72 percent) of the 
respondents believed that the pay program should be continued. (Kellor & Milanowski, 1999) 

Most of the open-ended responses to the guestion why people chose not to participate 
in the knowledge and skill pay system the first year revolved around process fairness issues, 
including the absence of specific procedures and policies relating to the evaluation system. 

Other freguently noted concerns had to do with the applicability and availability of the pay 
program to veteran teachers and the negative effect the new pay program would have on 
individual's base pay. These concerns, along with the other sentiments expressed through the 
survey and other feedback mechanisms, were all influential in determining the features of the 
performance pay program and evaluation system when it was extended to veteran teachers the 
following year. 

Survey results for the second year of operation (1999-2000) suggested that staff had 
become much more comfortable with the knowledge and skill standards. In general, a higher 
percentage of respondents indicated that they understood and accepted the standards and felt 
the process was fair. For example, 71 percent (vs. 44 percent) agreed that they knew what to do 
to increase their knowledge and skills, and 73 percent (vs. 47 percent) agreed that they had a 
good understanding of how the knowledge and skills were measured. The perception of fairness 
of the evaluation process also improved, with 72 percent (vs. 28 percent) agreeing that the way 
the knowledge and skills were established was fair, and 74 percent (vs. 28 percent) agreeing that 
the way the knowledge and skills were measured was fair. D espite these improvements, less than 
half (46 percent) agreed that overall, the assessment process was fairly run, and only about two- 
thirds agreed that they got a fair evaluation and that it was accurate. 

As part of an effort to examine the assessment process fairness and validity perception, 
the second year survey included specific guestions about the evaluator. In general, respondents 
agreed that evaluators were knowledgeable and provided positive recognition. They were less 
likely to agree that the evaluators communicated expectations, gave timely feedback, and 
provided coaching. Because it is not clear that these areas were explicitly included as part of the 
evaluation process, in retrospect it is not surprising that they were identified as areas of concern. 

The responses to guestions about the pay program itself were similar to the previous 
year. Most respondents felt that the pay amounts were motivating, but only a minority agreed 
that the pay system influenced their professional development choices. Fewer respondents than 
the previous year (64 percent vs. 83 percent) agreed that they made an effort to develop the 
knowledge and skills rewarded by the program. Because the program was expanded to include 
more experienced teachers, these responses could reflect an overall higher degree of confidence 
about one's teaching skills and less of a perception that changes in practice were needed to meet 
the teaching standards. The expansion of the program to include more veteran teachers did not 
seem to have a negative effect on how the pay system was perceived overall. 

The relatively minor changes that were made to the pay and evaluation programs after 
the second year reflected the generally high level of acceptance of the pay and evaluation 
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programs by staff. The creation of the Peer Assistance and Review Committee that began to 
operate during the third year probably was the most obvious response to the ongoing concerns 
about assessment process fairness and validity, and the responses to the survey after that year 
(see below) seemed to suggest that formalizing responsibilities for the oversight of the 
performance evaluation process paid off. (Milanowski & Kellor, 2000) 

After the third year (2000-2001), a large majority of respondents indicated that they 
understood the system, accepted the teaching standards, and found the process and results fair. 
However, while 76 percent agreed that their own evaluation was fair, only 65 percent felt it was 
accurate. Some persistent areas of concern continued, with just over half of the respondents 
agreeing that they had the professional development opportunities needed to develop the 
knowledge and skills, and 59 percent agreeing that they had the opportunity to develop the 
skills. Evaluators were perceived somewhat more favorably than in previous years, with 
about 80 percent of the respondents agreeing that the evaluators were knowledgeable about 
good teaching, but only about 40 percent agreeing that evaluators did a good job of 
communicating expectations and gave coaching that was needed to improve one's teaching. 
Respondents were split almost egually in terms of how they perceived the timeliness of 
feedback and whether the feedback helped to improve their teaching. Thus, although the 
respondents' perceptions of the evaluation process seemed to have improved in the aggregate 
over previous years, there were still lingering areas of concern. 

The responses to guestions about the pay program were similar to the previous year. 
Most respondents felt the pay amounts were motivating and fair, and for the first time, a 
majority of respondents indicated that the pay system influenced their professional development 
choices and that they made real changes to their teaching practice to receive the additional pay. 

A higher proportion of respondents (87 percent vs. 64 percent) also agreed that they made an 
effort to develop the knowledge and skills rewarded by the pay system. These are noteworthy 
responses in that they suggest that the pay system was beginning to have a direct influence on 
teachers' behavior. 

The responses to guestions about the effect of performance pay overall were similar to 
previous years' responses. A large majority of respondents agreed that performance pay had 
encouraged teachers to focus on improving instruction, improving student achievement, and to 
help each other develop skills. However, about a quarter agreed that performance pay had 
reduced the sense of community at the school, had hurt morale and there was more rivalry 
among teachers as a result of the performance pay program. About a third of the respondents 
believed that teachers resented other teachers who received more performance pay. These 
responses suggested that although a majority of V aughn's staff had a generally positive reaction 
to the performance pay program, there clearly were areas of concern and the pay and evaluation 
programs were not universally accepted within the school. (Milanowski & G allagher, 2001) 

The survey conducted after the fourth year (2001-2002) of the new pay and evaluation 
programs did not allow direct comparison of the prior years' responses because many of the 
questions were changed to conform to similar questions being used at other CPRE-UW 
research sites. However, broad themes can be compared, and overall, responses were similar to 
previous years. The changes to the teacher evaluation system seemed to have positively affected 
teachers' perceptions of it. About three-quarters of the respondents continued to agree that: the 
processes and procedures of the evaluation system were fair; the respondent's own evaluation 
was conducted in a fair way; the evaluation process was run fairly; the evaluation scores the 
respondent received were fair, and the evaluation process helped the respondents improve their 
teaching. However, there was a noticeable decrease from the previous year (77 percent vs. 91.3 
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percent) in those who agreed that working towards improving one's performance on the scoring 
matrix would really help teachers teach better. 

In general, respondents reported that they had a good understanding of the scoring 
matrix and of the evaluation system itself. Almost two-thirds of the respondents agreed that 
professional development opportunities were available to develop the knowledge and skills 
rewarded by the pay system, and more than three-quarters agreed that they had the opportunity 
to develop the knowledge and skills that she or he needed to do well. This suggested that 
although professional development opportunities were not always perceived as being available, 
the majority of respondents apparently found ways to develop the necessary knowledge and 
skills. 

Respondents' attitudes towards the evaluation process generally were favorable, 
although all aspects of the peer evaluator part of the process were perceived less favorably than 
the administrator parts of the process. The reasons for this disparity were not clear. For 
example, it could be that feedback and ratings by peers inherently are perceived differently by 
teachers than feedback and ratings by administrators, but that can not be determined without 
thoroughly reviewing the evaluation process to identify whether there were real differences 
between the two categories of evaluators. 

New teachers appeared to be somewhat more positive about the influence that 
performance pay has on teachers' behaviors and actions. They were more likely to agree that 
performance pay encouraged teachers to focus on improving instruction, that it encouraged 
teachers to help each other develop their knowledge and skills, and that it encouraged teachers 
to focus on improving student achievement. 

The effect of knowledge and skill-based pay on professional development activities and 
actual changes to instruction was reported as somewhat less than the previous year. As the 
school's staff became more familiar with the teaching expectations and more experienced in 
teaching in that manner, they might have felt less of a need to change their instruction. Vaughn 
also tried each year to offer school- wide professional development activities that supported 
development of the desired knowledge and skills, so perhaps staff did not feel as great a need to 
identify such professional development on an individual basis. 

D espite the bottom-up origins of the new pay system, the perception that it reduced the 
sense of community among teachers continued, with 15 percent (vs. 27% the prior year) of 
veteran teachers agreeing with that statement. Nonetheless, a strong majority of respondents 
agreed that the performance pay system should be continued, suggesting that the performance 
pay system is relatively well accepted by the school's teachers. 

0 f special interest to Vaughn administrators the fourth year was whether teachers were 
concerned about the potentially varying income that they could receive each year under the 
performance pay system. T wo questions addressed this concern. T eachers were asked how 
much more or less stable they think their future income is under V aughn's pay system, 
compared to Los Angeles Unified School D istrict's traditional salary system. About a quarter of 
veteran staff felt that their income was somewhat or much less stable, whereas less than ten 
percent of new staff responded in that manner. At the same time, 43 percent of new teachers 
and 30 percent of veteran teachers felt that their income was about the same, and about half of 
both new and veteran teachers felt that V aughn's pay system was somewhat more or much 
more stable than the Los Angeles district's. (Kellor & Milanowski, 2002) 

After the fifth year (2002-03), survey results relating to pay satisfaction continued to 
show that the performance pay system seemed to have more influence on new teachers' 
decisions to work at V aughn than it did for veteran teachers, with 57 percent of new teachers 
agreeing that the pay system was an important reason why they chose to teach at V aughn, 
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compared to 25 percent of veteran teachers who agreed with that. All of the new teachers who 
responded agreed that they have the opportunity to make more at V aughn than they do 
elsewhere, compared to just over half of the veteran teachers who agreed. Finally, while 79 
percent of the new teachers agreed that the performance pay system made working at V aughn 
more attractive to them, only 49 percent of the veteran teachers agreed with that statement. It is 
hard to determine whether the pay system's appeal to new teachers reflected primarily the 
structural differences between the traditional pay scales and the performance pay system, which 
has a competitive starting base rate and offers more opportunities for new teachers to earn 
additional pay, or if it reflected philosophical differences between how new teachers and veteran 
teachers view their pay opportunities. (Kellor & Milanowski, 2003) 

As noted previously, significant changes were made to the PAR process and the 
performance pay program for the 2003-04 school year. Overall, the survey responses showed 
mostly favorable attitudes towards the new PAR system, and most areas showed an increase in 
the percentage of favorable responses. O ne potential area of concern was in response to the 
training on the new PAR system. Just over half felt they received enough training, and about a 
quarter were neutral about the amount of training provided. G iven the direct linkage of the 
evaluation system to the performance pay system, it is important to provide adequate training 
on the PAR system, and the responses suggest the need to improve the training. However, the 
responses to questions about the fairness of the evaluation process and the clarity of 
expectations were generally favorable, and 70% agreed that the new evaluation system was an 
improvement over the previous one. If the increases in favorable responses are attributable to 
the changed rubrics, it could suggest that teachers are more comfortable thinking of their 
teaching in a more general and holistic way, rather than the more specific strategies and 
concepts that were associated with the content-specific rubrics used in previous years. Overall, 
the evaluation system continued to have a high degree of acceptance, with just over three- 
quarters of respondents indicating that the school has an excellent performance evaluation 
system. 

O ne of the reasons cited for changing the teacher evaluation rubrics from content 
specific to more general teaching domains was to address concerns that only particular "types" 
of teaching were recognized under the previous system. Thus, a new question asked whether 
changes to the PAR system in 2003-04 made it easier for the respondent to receive the score 
needed for performance pay; 60% agreed that it was easier, with 29% neutral. An increase in the 
number of teachers receiving performance pay could be an indicator of whether the perception 
of the new PAR being "easier" than the old one was valid, although the school has not yet 
conducted an analysis of the actual pay pay-outs under the two systems. 

Overall, both veteran and new staff had positive attitudes about the performance pay 
system. The responses of new staff were supportive of the pay program but the extent to which 
it influenced their decision to teach at Vaughn wasn't clear. For example, although 76% of the 
new teachers agreed or strongly agreed that they "would have the opportunity to make more 
money teaching at Vaughn than . . . elsewhere," only 34% agreed that "the performance pay 
system is an important reason [they] chose to teach at V aughn." For veteran teachers, the 
responses generally were about the same as in previous years, although fewer teachers agreed 
that performance pay increases rivalry between teachers, and that teachers who get more 
performance pay are resented by those who don't do as well. The majority of veteran teachers, 
73%, agree that the system should be continued. (Kellor & Milanowski, 2004) 
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Wrapping It Up: Lessons from Vaughn's Experience 

A standard phrase in any version of Vaughn's history is that it is a "conversion 
charter school." After six years of a knowledge and skill-based pay program and associated 
standards-based teacher evaluation system, it could rightfully be called a "conversion 
performance pay school," with all staff now paid on performance pay systems. Although 
Vaughn is an exceptional school in many ways, its transformation to a performance pay-driven 
organization has not been without some difficulties, some of which the pay plan advocates 
expected and others which were surprises. 0 ne of the reasons the V aughn staff pursued a new 
pay system was to use the advantages of flexibility and innovation that being a charter school 
gave them, so that the lessons they learned from their pay experiment could be shared with 
others who might be in settings where the ability to take risks and be innovators was less 
supported. This last section highlights key points that could help make the journey to 
knowledge and skill-based pay or standards-based teacher evaluation a bit less arduous for 
others who are ready to move towards those education reforms. 

Expect changes, but don't change too much or too often. From the outset, 

V aughn's pay plan developers and designers of the teacher evaluation system knew that changes 
would be necessary, both to refine rough edges as well as to address unforeseen complications 
or situations. Some years saw more significant changes than other years, but efforts were made 
to make only those changes that were essential each year. This was done for several reasons, 
including the ability to accurately project costs for future years, to help teachers gain experience 
and trust in the new programs, and to be certain that only those changes that truly were 
necessary were made. 

E ncourage involvement of representative people in designing the system, but 
recognize the need fora "change champion" at times. Vaughn's governance structure 
reguires teacher, administrator and parent involvement in almost every important decision 
relating to the school. The knowledge and skill pay plan and new evaluation system, although 
not initially designed by a formally designated group, followed the same inclusive principles. 

This was important because it ensured that the people most directly affected by the new 
programs— the teachers— were directly involved in key decisions about the programs. The 
governance structure included an approval process that ensured that broader concerns of all 
teachers, and more important, of students, would be considered before the pay and evaluation 
programs were put into place. The principal played an important role throughout this, providing 
support, direction and encouragement, along with ensuring that any data or assistance the 
designers needed was provided. 0 n the flip side, the principal believed that she might have 
stepped back too soon, or inappropriately, at a time when the designers were facing 
controversial and emotional issues. Although she did this to empower the teachers and make it 
clear they had authority to design the programs, her absence of leadership at a critical time 
might have made the expansion of the program to veteran staff more difficult. However, it also 
is possible that the heated discussions that took place when the program was designed and the 
"true colors" that were exhibited at this time could have had a cathartic effect in that people's 
positions and concerns were made clear. Others involved in designing a new compensation 
system should consciously consider the positive and negative implications of a leader stepping 
away from a leadership role at critical points. 

Even in a collegial setting, expect some "fallout" from changes for which 
there is no objective way to determine what's "best" or "fair." According to interviews and 
anecdotal comments made to researchers, the Vaughn staff generally considered themselves to 
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be a collegial group that regularly put students' interests ahead of other interests, evidenced in 
part by the fact that the staff who remained at V aughn when it became a charter school made a 
conscious choice to stay rather than return to a district position. Y et when it came to bottom- 
line decisions that had to do with individual pay levels, some of that collegiality seemed to 
disappear and differences began to emerge about how people felt about important things like 
how individual effort should be recognized and compensated, and if describing the elements of 
"good instruction" took away from individual teacher autonomy and diminished the "art of 
teaching." This probably should not have been a surprise, especially when changing something 
as long-standing as the single salary schedule, but it seemed to take many people by surprise. 
Over time, Vaughn's sense of collegiality and morale seem to be recovering, perhaps aided by 
the gradual attrition of those staff who did not agree with the changes. Whether the school's 
sense of community would have been restored if a significant number of performance pay 
"dissenters" had stayed at the school is unknown, but places contemplating changes to their pay 
systems should consider the possibility that there will be splits among the staff and develop 
strategies to rebuild the community. 

Communication is essential. Encourage the free flow of information, do what 
you can to ensure that accurate information is being presented, and establish and 
document clear, written policies and procedures so that the rules and expectations are 
known from the outset Vaughn's pay plan and evaluation system designers did an admirable 
job of identifying important issues in advance and addressing them, either by a specific decision 
or by establishing policy directions to guide future decisions and make the programs' goals clear 
to teachers. Y et in some important areas the planners knowingly proceeded without specifying 
details (e.g., implementing the performance pay program at the beginning of the school year and 
working out the details of the evaluation process during the first several months of school) and 
in other areas (e.g., creating descriptions of performance levels in the evaluation system) its 
effort fell short. In Vaughn's environment, the lack of specificity created tension the first year 
and was a key priority for the second year of the program, but for many schools, the lack of 
specificity in areas as critical as these could have been a fatal flaw. The quick response to resolve 
the problems probably helped give the new programs credibility and to gain teachers' trust, but 
implementing a program without knowing and / or communicating details and policies 
introduces an even greater element of risk to the program's success and is not the preferred 
method for an effective implementation. 

T ransition details from one pay system to another are important; minimizing 
risk to veterans while meeting the goals of a new program can be a delicate balance to 
reach. Vaughn's approach to implementing a new pay system was to begin with the group of 
teachers who were likely to have less invested in the current pay system, i.e., new and early 
career teachers. Although new teachers were required to be part of the new pay system, even 
early career teachers were given the option to stay under the existing pay plan or to become part 
of the pilot. When the program was offered to veteran teachers sooner than originally planned, 
attention to the details of the transition were an important part of the design process, from 
establishing base rates that were somewhat reflective of the veterans' pay under the traditional 
salaiy schedule, to setting up a method to essentially keep a teacher's monthly pay close to its 
previous level on at least a temporary basis so that there would not be a precipitous drop in 
income before the performance pay was paid out. Another way that V aughn dealt with 
transition issues was in structuring the pay system for veteran teachers so that some vestiges of 
experience were retained in the salary structure, both in the form of annual increases up to some 
point and by creating advanced performance levels that realistically would only be attainable by 
the more experienced teachers. (Vaughn's approach was still quite aggressive in that a teacher 
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could end up with net compensation lower than the previous compensation, but given the 
overall confidence in teachers' ability to meet the performance pay standards the risk of losing 
net pay did not seem as real as it might have in other school settings.) 

Accommodating veterans' interests in the new salary structure in a way that was still 
compatible with the long-term goals of the performance pay program and the evaluation system, 
and identifying transition issues and providing ways to deal with them, very likely made the 
transition to the new systems more attractive and easier than if the pay system completely 
ignored the lingering influence of the traditional pay structure. 

In brief, Vaughn's multi-year experience with its knowledge and skill-based pay and 
standards-based teacher evaluation system provide useful insights to others who are considering 
or are in the midst of teacher compensation and evaluation reform projects. Although the 
school's status as a charter school makes it difficult to generalize the exact circumstances of its 
compensation and evaluation reforms, the lessons derived from its experiences the past several 
years are useful regardless of the context. The evidence of the predictive ability of its teacher 
evaluation system on student achievement (that the higher the evaluation score, the higher the 
learning gains produced in that teacher's classroom) is exciting and the fact that the standards- 
based evaluation system was introduced to meet the needs of the knowledge and skill-based pay 
program lends credence to the recommendation to aligning internal systems and resources so 
they support the same goals. 


Note 

This article was prepared for the Consortium for Policy Research in Education, Wisconsin 
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and the Wisconsin Center for Education Research, School of Education, University of 
Wisconsin-Madison (Grant No. OERI-R308A 60003). The opinions expressed are those of the 
authors and do not necessarily reflect the view of the National Institute on Educational 
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Appendix A: Vaughn Pay Summary 1998-2003 



1998-99 

1999-00 

2000-01 

2001-02 

2002-03 

Year 1 

30,000 

31,500 

31,500 

33,150 

33,150 

Year 2 

31,000 

32,500 

32,500 

34,150 

34,150 

Year 3 

32,000 

33,500 

33,500 

35,150 

35,150 

Year 4 1 

33,000 

34,500 

34,500 

36,150 

36,150 

Year 5 

34,000 

36,500 

36,500 

38,150 

38,150 

Year 6 2 


38,000 

38,000 

41,650 

41,650 

Year 7 


39,500 

39,500 

43,150 

43,150 

Year 8 


41,000 

41,000 

44,650 

44,650 

Year 9 


42,500 

42,500 

46,150 

46,150 

Year 10 


44,000 

44,000 

47,650 

47,650 

Year 11+ 


45,500 

45,500 

49,150 

49,150 

Y ear 15+ 


47,500 

47,500 

51,150 

51,150 







CA credential 

1,000 

2,000 

2,000 

2,000 

2,000 

MA 3 

1,000 

2,000 

2,000 

2,000 

2,000 

NBPTS 

4,000 

4,000 

4,000 

4,000 

4,000 







Level 1: (depth in essentials; requires averaged 
self-peer-administrator rating of 2.5 & clear credential) 

Literacy 

1,300 

1,300 

1,300 

1,300 

1,300 

ESL, Sheltered 
English 4 

1,300 

1,300 

1,300 

1,300 

1,300 

Technology 

400 

400 

- 

- 

- 

Mathematics 



1,000 

1,000 

1,000 

Special Ed 
inclusion 

300 

300 

300 

300 

300 

Classroom mgt 


100 

100 

300 

300 

Lesson planning 


100 

100 

300 

300 

Level 2: (depth +breadth after achieving 3.0in all Level lareas) 

Mathematics 


1,000 

-- 

- 

- 

Social studies 


800 

800 

900 

900 

Science 


800 

800 

900 

900 

English learner's 
support (BCLAD) 


2,500 

2,500 

2,500 

Strand 1: 2,500 
Strand 2: 2,000 
Strand 3: 1,000 

2,500 

Strand 1: 2,500 
Strand 2: 2,000 
Strand 3: 1,000 

Technology 


- 

500 

500 

500 

Physical education 


- 

- 

500 

500 

Arts 


500 

500 

500 

500 

Level 3: 






If average rubric 
>3.5 in level 2: 


4,000 

4,000 

4,500 

4,500 

Level 4: 

Distinguished 

teacher 






After 5 


- 

- 

2,000 

2,000 


1 From year 4 on, must have an evaluation of “average” or better to receive annual pay increase 

2 To receive Year 6 and above pay levels, must possess a clear California elementary teaching credential 

! Beginning 1999-2000, paid for holding a Master's degree in education, OR for having 30 units after obtaining 
clear credential 

4 Called “English language development” beginning 1999-2000. 
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1998-99 

1999-00 

2000-01 

2001-02 

2002-03 

consecutive years 
at Level 3 






Contingency- 

based 






Student attendance 

25 

0 

250 

250 

250 

250 

Student discipline 

25 

0 

250 

250 

250 

250 

Parent partnership 

15 

0 

250 

250 

250 

250 

T earning efforts 

15 

0 

250 

250 

500 

500 

Outcome-based 






Schoolwide 

achievement 

1,5 

00 





Schoolwide 

redesignation 

25 

0 





Grade- level 
achievement 

50 

0 





Individual class 
achievement 

50 

0 





API (state reform) 


1,50 

0 

1,500 

2,000 

2,000 

Expertise-based 






Committee chair 

50 

0 

500 

500 

500 

500 

Clan leader 

50 

0 

500 

500 

750 

750 

Faculty chair 

50 

0 

500 

500 

500 

500 

Mentor 

50 

0 

500 

500 

500 


Staff dev leader 

Per 

ses 

sio 

n 

$25/ 

hour 




Parent ed leader 

Per 

ses 

sio 

n 

$25/ 

hour 




Afterschool 

50 

0 

500 

500 

$25/ hour 

$25/ hour 

Extended year 

Ex 

tra 

mo 

nth 

pay 

Per 

polic 

y 

Per 

policy 

Per policy 

Per policy 

Peer reviewer 




500 

750 

Gainsharing 






Substitutes 

25 

0 

250 

250 

250 

250 

Stock options 
(minimum) 

"" 

1,00 

0 

1,000 

1,000 

1,000 
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Appendix B: Vaughn N ext Century Learning Center 
Performance-Based Pay Plan Effective 2003-04 School Year 



Base 

Salary 

Base 
salary, 
with CA 
credential 


C ontingency-based 



Year l 1 

33,150 

38,900 


Student attendance 

At least 95%: $250 

>98%: $1,000 

Year 2 

34,150 

39,900 


API (state reform) 

$2,000 


Year 3 

35,150 

40,900 


Expertise-based 



Year 4 

36,150 

41,900 


Committee chair 

$500 


Year 5 

38,150 

43,900 


Clan leader 

$750 


Year 6 2 


45,400 


Faculty chair 

$500 


Year 7 


46,900 


Peer Reviewer 

$1,000 


Year 8 


48,400 





Year 9 


49,900 


Gainsharing 



Year 10 


51,400 


Substitutes 

$250 


Year 11+ 


52,900 


C ontingency-based 



Year 15+ 


54,900 


Student attendance 

At least 95%: $250 

>98%: $1,000 

MA in Ed OR 
30 units after 
clear 

credential 

$2,000 


NBPTS 

$4,000 

Level L (depth in essentials; reguires averaged self-peer-administrator rating of 3.0) 

Domain 1: 
Planning and 
preparation 

$1,000 


Domain 2: 
Classroom 
environment 

$1,775 

Domain 3: 
Instruction 

$1,775 

D omain 4: 

Professional 

responsibility 

$1,000 

Total Level 1: 

$5,550 


Level 2: Clear Credential T eachers 

(depth + breadth after achieving 3.0 in all Level 1 areas and average is >3.5) 


$5,500 


L evel 3: D istinguished T eacher (after 8 out of 10 consecutive semesters at L evel 2) 

$ 2,000 

'Teachers with satisfactory evaluation (2.5) will advance yearly 

2 T o receive Year 6 and above pay levels, must possess a clear California elementary teaching 
credential. 
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